Endoscopy system and method therefor

ABSTRACT

A capsule endoscope is swallowed by a patient, to capture images from inner wall surfaces of patient&#39;s digestive tract, while detecting imaging positions of the capsule endoscope. A doctor observes the images captured by the capsule endoscope, to select an aimed point image that contains a portion suspected of a lesion, and some images that represent pass points on an insertion route of a balloon endoscope. While a probing tip of the balloon endoscope is being inserted into the patient for the sake of thorough examination of the suspected portion, a degree of similarity between an image captured by the balloon endoscope and an image of a destination point, which is one of the pass points and the aimed point, is detected to judge by the similarity whether the probing tip has reached the destination point or not. So the doctor can get the probing tip to the aimed point easily.

FIELD OF THE INVENTION

The present invention relates to an endoscopy system and a method forthe endoscopy system, wherein a capsule endoscope is used for a primaryexamination and a flexible endoscope is used to capture images fromthose body sites which are determined to need a thorough examination asa result of the primary examination.

BACKGROUND OF THE INVENTION

Endoscopy with a capsule endoscope has recently been put into practicaluse. The capsule endoscope has its components, including an imagingdevice and a light source, integrated in a micro capsule. A patientfirst swallows the capsule endoscope so that the imaging device capturesimages from internal body sites, i.e. internal surfaces of patient'stracts, while the light source is illuminating those sites. Image datacaptured by the imaging device is transmitted as a radio signal to areceiver that is attached to the patient. The image data is sequentiallyrecorded on a storage medium like a flash memory, which is provided inthe receiver.

In parallel to the imaging of the body sites by the capsule endoscope,the position of the capsule endoscope inside the patient is detected.For example, JPA 2005-192880 and JPA 2007-236700 suggest sending out aradio wave from the capsule endoscope and detecting the strength of theradio wave received on an antenna, which may be mounted on a shieldingshirt or the like that the patient wears. Then data on the position ofthe capsule endoscope is derived from the strength of the receivedelectric wave, and is recorded in association with the image data of theinspected sites on the storage medium.

To complete the endoscopy with the capsule endoscope, the receiver isconnected to an information managing apparatus like a workstation via anUSB cable or the like, so that the whole image data stored in thereceiver is taken into the information managing apparatus. On the basisof the image data taken into the information managing apparatus, adoctor has the captured images displayed on a monitor to interpret them.When the doctor finds any suspected part, i.e. such a part that lookslike a lesion, in some images, the doctor takes images from thesuspected part with a flexible endoscope like a balloon endoscope, tomake a complete examination.

If the suspected part is found in those images taken from inside a smallintestine, a probing tip of the balloon endoscope is inserted into thesmall intestine to take images of an aimed point by an imaging devicethat is built in the probing tip. The aimed point is where the suspectedpart may exit, and its position is located in advance on the basis ofthe position data of the capsule endoscope that is recorded inassociation with the image data of the suspected part. Image signalsfrom the imaging device are sent to a processor that is connected to theballoon endoscope. Then the processor processes the image signals todisplay an endoscopic image on a monitor, so that the doctor interpretsthe displayed image for diagnosis.

To insert the probing tip of the balloon endoscope into the smallintestine to reach the aimed point, the small intestines are drawn inwith a balloon that is provided on the probing tip. As a result, thedoctor tends to lose track of the aimed point because the aimed point inthe drawn small intestines does not coincide with one indicated by theposition data that was detected by the capsule endoscope in the normalcondition of the small intestines. So it takes a pretty long time tosearch for the aimed point, which elongates the total time taken for theendoscopy and thus increases the load on the patient.

SUMMARY OF THE INVENTION

In view of the foregoing, a primary object of the present invention isto provide an endoscopy system and an endoscopy method, which make itpossible to detect a relative position of a probing tip of a flexibleendoscope such as a balloon endoscope inside a test body while theprobing tip is being inserted into the test body, on the assumption thatthe endoscopy system and the endoscopy method use a capsule endoscopeswallowed by a test body to capture first kind of images from internalportions of the test body and a flexible endoscope having a flexibleinserter with an imaging device, which is inserted in the test body tocapture second kind of images by the imaging device when the doctorfinds it necessary to make a thorough examination of an aimed pointinside the test body as a result of interpretation of the first kind ofimages.

The endoscopy system of the present invention comprises an aimed pointselection device for selecting an aimed point image that contains theaimed point from among the first kind of images in response to anoperation by the doctor; a similarity detection device for detectingsimilarity between the aimed point image and the second kind of imagesas captured by the imaging device while the flexible inserter is beingmoved toward the aimed point; a position information obtaining devicefor obtaining information on a relative position of the imaging deviceof the flexible endoscope inside the test body on the basis of thesimilarity detected by the similarity detection device; and a displaydevice for displaying the information on the relative position of theimaging device inside the test body.

According to a preferred embodiment, the endoscopy system furthercomprises a pass point selection device for selecting at least a passpoint image from among the first kind of images, the pass point imagebeing representative of a pass point on a route from an inlet of theflexible inserter to the aimed point, wherein the similarity detectiondevice further detects similarity between the pass point image and thesecond kind of images while the flexible inserter is being inserted intothe test body, and the position information obtaining device obtains theinformation on the relative position of the imaging device to the passpoint or to the aimed point on the basis of the similarity between thepass point image and the second kind of images or the similarity betweenthe aimed point image and the second kind of images, respectively.

Preferably, the display device displays information as to whether theimaging device has reached the pass point and the aimed point as theinformation on the relative position of the imaging device.

Preferably, the similarity detection device detects the similaritybetween the pass point image and the second kind of images bycalculation using image characteristic values of the pass point imageand the second kind of image, and the similarity between the aimed pointimage and the second kind of images by calculation using imagecharacteristic values of the aimed point image and the second kind ofimage.

According to a preferred embodiment, the endoscopy system furthercomprises a first spectral image producing device for producing spectralimages of appropriately selected spectral frequency bands from the aimedpoint image and the pass point image respectively; and a second spectralimage producing device for producing a spectral image from each of thesecond kind of images so that the spectral image has the same spectralfrequency band as the spectral image of the pass point has while theimaging device of the inserter is moving toward the pass point, and thatthe spectral image has the same spectral frequency band as the spectralimage of the aimed point has while the imaging device of the inserter ismoving from the pass point toward the aimed point, wherein the firstimage characteristic value taking device takes the image characteristicvalues from the spectral images of the aimed point and the pass point,whereas the second image characteristic value taking device takes theimage characteristic values respectively from the spectral images of thesecond kind of images.

Preferably, the image characteristic values taken by the first andsecond image characteristic value taking devices represent blood vesselpatterns in the internal portions of the test body.

According to another preferred embodiment, surface asperities of theinternal portions of the test body are detected as the imagecharacteristic values. In that case, the capsule endoscope preferablycomprises a number of light sources, which are disposed at differentpositions and sequentially emit light to illuminate the same portioninside the test body. The capsule endoscope captures a correspondingnumber of images to the number of light sources from the same portionsynchronously with the sequential emissions of the light sources towardthe same portion. Then, the first image characteristic value takingdevice estimates the surface asperities of the pass point and the aimedpoint on the basis of images captured from the pass point and imagescaptured from the aimed point respectively. On the other hand, theinserter of the flexible endoscope is provided with a plurality ofillumination windows on different sides of the imaging device, toproject illumination light sequentially from one illumination windowafter another toward the same portion inside the test body, and theimaging device of the flexible endoscope captures a corresponding numberof images to the illumination windows from the same portionsynchronously with the sequential projection of illumination light fromthe illumination windows toward the same portion. Then, the second imagecharacteristic value taking device estimates the surface asperities ofthe same portion on the basis of the images captured by the imagingdevice of the flexible endoscope from the same portion.

An endoscopy method of the present invention comprises steps ofselecting an aimed point image that contains the aimed point from amongthe first kind of images before inserting the flexible inserter into thetest body; detecting similarity between the aimed point image and thesecond kind of images as captured by the imaging device while theflexible inserter is being moved toward the aimed point; obtaininginformation on a relative position of the imaging device of the flexibleendoscope inside the test body on the basis of the similarity betweenthe aimed point image and the second kind of images; and displaying theobtained information on the relative position of the imaging device.

According to the present invention, information on the relative positionof the imaging device of the flexible endoscope inside the test body isobtained on the basis of similarity between the aimed point image thatis selected from among the first kind of images as captured by thecapsule endoscope, and the second kind of images as captured by theflexible endoscope. Therefore, even while the relative position of theaimed point inside the test body varies during the endoscopy with theflexible endoscope from the relative position of the aimed point duringthe endoscopy with the capsule endoscope, the present invention ensuresdetecting whether the imaging device of the flexible endoscope hasreached the aimed point or not. As a result, the doctor can get theimaging device of the flexible endoscope to the aimed point in a shortertime than conventional, which contributes to shortening the total timeof inspection and thus reducing the load on the patient.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the present invention willbe more apparent from the following detailed description of thepreferred embodiments when read in connection with the accompanieddrawings, wherein like reference numerals designate like orcorresponding parts throughout the several views, and wherein:

FIG. 1 is a schematic diagram illustrating a capsule endoscopy system asa component of an endoscopy system according to an embodiment of thepresent invention;

FIG. 2 is a schematic diagram illustrating an electronic endoscopysystem as another component of the endoscopy system;

FIG. 3 is a block diagram illustrating an electric structure of thecapsule endoscopy system;

FIG. 4 is a block diagram illustrating an electric structure of areceiver;

FIG. 5 is a block diagram illustrating an electric structure of a firstprocessor as a component of the capsule endoscopy system;

FIG. 6 is an explanatory diagram illustrating data stored in a datastorage of the first processor;

FIG. 7 is an explanatory diagram illustrating an aimed point and passpoints on an endoscope insertion route, which are selected by a doctorwith reference to images captured by a capsule endoscope;

FIG. 8 is an explanatory diagram illustrating a point image selectionscreen for the doctor to select image files corresponding to the aimedpoint and the pass points;

FIG. 9 is a block diagram illustrating an electric structure of a secondprocessor as a component of the electronic endoscopy system;

FIG. 10 is an explanatory diagram illustrating a point detection screenthat shows whether the tip of the balloon endoscope has reached adestination point or not, in a stage where the tip has not yet reachedthe destination point;

FIG. 11 is an explanatory diagram illustrating the point detectionscreen in a stage where the tip has reached a first pass point and asecond pass point is set to be the next destination point;

FIG. 12 is a flow chart illustrating a process of extracting imagecharacteristic values from respective image data of an aimed point andpass points, which are selected by the doctor;

FIG. 13 is a flow chart illustrating a process of detecting which pointon the endoscope insertion route the tip of the balloon endoscope hasreached;

FIG. 14 is an explanatory diagram illustrating an aimed point and passpoints on an endoscope insertion route from the patient's anus;

FIG. 15 is an explanatory diagram illustrating point imagecharacteristic values used in an endoscopy system according to a secondembodiment of the present invention;

FIG. 16 is an explanatory diagram illustrating a point detection screenaccording to the second embodiment;

FIG. 17 is a sectional view of a capsule endoscope for use in anendoscopy system according to a third embodiment of the presentinvention;

FIG. 18 is an explanatory diagram illustrating how the capsule endoscopeof the third embodiment captures a couple of images from a portion whileswitching illumination light between a first light source and a secondlight source;

FIG. 19 is an explanatory diagram illustrating a process of estimatinginformation on surface asperities of the subject on the basis of thecouple of images captured from the same subject by the capsule endoscopeof the third embodiment;

FIG. 20 is a front view of a face end of a tip of an inserter of aballoon endoscope for use in the third embodiment;

FIG. 21 is a sectional view of another capsule endoscope for use in thethird embodiment;

FIG. 22 is a sectional view of still another capsule endoscope for usein the third embodiment; and

FIG. 23 is a perspective view of a tip of another balloon endoscope foruse in the third embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

As shown in FIGS. 1 and 2, an endoscopy system 2 consists of a capsuleendoscopy system 3 and an electronic endoscopy system 4. In theendoscopy system 2, an endoscopy of a patient or test body 10 is firstmade using the capsule endoscopy system 3 and, if any suspected partthat can be a lesion or the like is found, a thorough examination of thesuspected part is made using the electronic endoscopy system 4.

The capsule endoscopy system 3 consists of a capsule endoscope 11 thatis swallowed into the patient 10, a portable receiver 12 carried aboutby the patient 10, and a workstation 13 that takes up images as capturedby the capsule endoscope 11 and displays the images for a doctor tointerpret them.

The capsule endoscope 11 captures images from internal walls of tracts,e.g. small bowels, of the patient 10, to send data of the capturedimages to the receiver 12 sequentially as a radio wave 14. The receiver12 is provided with a liquid crystal display (LCD) 16 for displayingvarious setup screens and an operating section 17 for setting up thereceiver 12 on the setup screens. The receiver 12 receives and storesthe image data as transmitted from the capsule endoscope 11 on the radiowave 14, which will be referred to as the CE image data hereinafter.

The transmission of the radio wave 14 between the capsule endoscope 11and the receiver 12 is carried out by way of antennas 18 and 20, whereinthe antenna 18 is mounted in the capsule endoscope 11, as shown in FIG.3, whereas the antennas 20 are mounted on a shield shirt 19 that thepatient 10 wears. Each of the antennas 20 has an electric field strengthsensor 21 built therein for measuring the field strength of the radiowave 14 from the capsule endoscope 11.

The workstation 13 is provided with a first processor 24, operatingmembers 25, including a keyboard and a mouse, and an LCD 26. The firstprocessor 24 is connected to the receiver 12, for example, through anUSB cable 27, to exchange data. The first processor 24 may be connectedto the receiver 12 through wireless communication like infraredcommunication. The first processor 24 takes up the CE image data fromthe receiver 12 during the capsule endoscopy with the capsule endoscope11 or at the end of the capsule endoscopy, to accumulate and manage theCE image data individually for each patient. Simultaneously, the firstprocessor 24 generates an endoscopic image based on the CE image data,which corresponds to the first endoscopic image as specified above inthe summary of the invention, and will be referred to as a CE imagehereinafter. The first processor 24 displays the CE image on the LCD 26,so a doctor interprets the CE images on the LCD 26.

The first processor 24 is also connected to a second processor 32 of theelectronic endoscopy system 4 through a LAN 29. When the doctor selectssome CE images by operating the operating members 25 of the firstprocessor 24, the first processor 24 extracts image characteristicvalues from the image data of the selected CE images and transmits theextracted characteristic values to the second processor 32 through theLAN 29.

In an embodiment shown in FIG. 2, the electronic endoscopy system 4 is aflexible endoscope for small-bowel examination, which consists of aballoon endoscope 31 that is inserted into the patient 10 through itsmouth or another inlet, the second processor 32, an illuminator 33,operating members 34, including a keyboard and a mouse, and an LCD 35.The balloon endoscope 31 is provided with a flexible inserter 37 that isinserted into the patient's body, a handle 38 joined to a base end ofthe inserter 37, and a universal cord 39 that is connected to the secondprocessor 32 and the illuminator 33.

A probing tip 37 a is joined to a distal end of the inserter 37. Asshown in FIG. 9, an objective lens 41 and an imaging device 42 aremounted in the probing tip 37 a, for capturing images from the internalwalls of the patient's tracts, i.e. the small bowels in this embodiment.Analog image signals output from the imaging device 42 are converted todigital image data, hereinafter referred to as the BE image data, andfed to the second processor 32 through the universal cord 39.Simultaneously, illumination light from the illuminator 33 is conductedthrough an optical fiber cable or the like, which is mounted in theinserter 37, to the probing tip 37 a and is projected from the probingtip 37 a toward the small bowel internal wall.

A bendable portion 37 b is provided behind the probing tip 37 a. Thebendable portion 37 b consists of a number of segments coupled to oneanother in such a manner that the bendable portion 37 b bends in anydirections as an angle knob of the handle 38 being operated to push andpull wires that are mounted in the inserter 37. Thereby, the doctor canorient the probing tip 37 a to any desirable direction inside the testbody by operating the angle knob. Behind the bendable portion 37 b isprovided a flexible soft portion 37 c.

A balloon 48 is mounted between the probing tip 37 a and the bendableportion 37 b. The balloon 48 is made of an elastically expandablematerial, e.g. latex rubber. A not-shown ventilator feeds air into andout of the balloon 48 through a not-shown ventilation tube that isprovided along inside the inserter 37 and the universal cord 39, causingthe balloon 48 to swell out and deflate. As known in the art, theprobing tip 37 a advances into small bowels by drawing the small bowelswith the pinch-and-swell movement of the balloon 48, to performsmall-bowel endoscopy with the balloon endoscope 31.

The second processor 32 produces an endoscopic image on the basis of theBE image data from the balloon endoscope 31, and displays the endoscopicimage on the LCD 35. Hereinafter the endoscopic image produced based onthe BE image data will be referred to as the BE image, which correspondsto the second endoscopic image as specified above in the summary of theinvention. Since the BE image is clearer than the CE image, theendoscopy with the balloon endoscope 31 is suitable for detailedexamination of a body part under suspicion of a lesion, which was foundby the endoscopy with the capsule endoscope 11. The second processor 32also detects a relative position of the probing tip 37 a of the balloonendoscope 31 inside the patient 10 on the basis of the imagecharacteristic values of the CE image data, which are fed from the firstprocessor 24 through the LAN 29, as will be described in detail withreference to FIGS. 10 and 11.

Now the capsule endoscope 11, the receiver 12 and the first processor24, which constitute the capsule endoscopy system 3, will be describedin more detail with reference to FIG. 3. The overall operation of thecapsule endoscope 11 is supervised by a CPU 50. The CPU 50 is connectedto a ROM 51, a RAM 52, an imaging driver 53, a modulator circuit 54, apower supply circuit 56 and an illuminator driver 57. Designated by areference numeral 140 in FIG. 3 is an attitude sensor that detects aphysical attitude of the capsule endoscope 11 inside the body of thepatient 10, which will be described later with respect to a secondembodiment.

The CPU 50 reads out necessary programs and data from the ROM 51 andexpands them on the RAM 52 to process the read programs sequentially.The imaging driver 53 is connected to an imaging device 60 and a signalprocessing circuit 61. The imaging device 60 is for example a CCD or aCMOS that captures an image of a subject, a body site or body part, asthe image is formed through an objective lens 59. The objective lens 59has an imaging field of 140 to 180 in front angle of view, and forms anomniazimuth image of the subject existing in the imaging field. Theimaging driver 53 controls the operation of the imaging device 60 andthe signal processing circuit 61 so as to capture an image at a givenframe rate and a shutter speed. Designated by a reference numeral O isan optical axis of the objective lens 59.

The signal processing circuit 61 processes the image signal, which isoutput from the imaging device 60, through correlated-double sampling,amplification and analog-to-digital conversion, to convert it to thedigital CE image data. The signal processing circuit 61 also subjectsthe CE image data to gamma correction and other image processingprocesses.

The modulator circuit 54 is connected to an output of the signalprocessing circuit 61, and an input of a sender circuit 63, which isconnected to the antenna 18. The modulator circuit 54 modulates a radiowave in accordance with the CE image data from the signal processingcircuit 61, and output the modulated radio wave 14 to the sender circuit63. The sender circuit 63 outputs the radio wave 14 to the balloonendoscope 18 after amplifying and band-pass filtering it. Thus, the CEimage data is transmitted from the capsule endoscope 11 to the receiver12 wirelessly.

The power supply circuit 56 supplies power from a battery 64 torespective components of the capsule endoscope 11. The illuminatordriver 57 drives an illuminator 65 under the control of the CPU 50 so asto illuminate the subject or target body site at a given light volumeduring the imaging.

As shown in FIG. 4, the overall operation of the receiver 12 iscontrolled by a CPU 67. A data bus 68 connects the CPU 67 to respectivecomponents of the receiver 12, including a ROM 69, a RAM 70, ademodulator circuit 71, an image processing circuit 72, a data storage73, an input interface (I/F) 74 and a position detector circuit 75. Thedata bus 68 is also connected to an LCD driver 76 for driving the LCD16, a communication interface (I/F) 78 for intermediating data exchangebetween the receiver 12 and the first processor 24 via a USB connector77, and a power supply circuit 80 for supplying power from a battery 79to the respective components of the receiver 12.

The CPU 67 reads necessary programs and data from the ROM 69 and expandsthem on the RAM 70 to process the read programs sequentially. The CPU 67also controls the respective components of the receiver 12 to work inaccordance with operational signals input through the operating section17. The demodulator circuit 71 is connected to an output of a receivercircuit 81, and an input of the receiver circuit 81 is connected to theantenna 20. The demodulator circuit 71 demodulates the radio wave 14 asreceived from the capsule endoscope 11 to be the original CE image data,and outputs the CE image data to the image processing circuit 72. Thereceiver circuit 81 outputs the radio wave 14 as received on the antenna20 after amplifying and band-pass filtering it.

The image processing circuit 72 processes the CE image data as decodedby the demodulator circuit 71, and outputs the processed CE image datato the data storage 73 while attaching ID information such as a filename to each individual image file of the CE image data. The datastorage 73 is for example a flash memory with a memory capacity of about1 GB. The data storage 73 stores the CE image data as output from theimage processing circuit 72. The input interface 74 gets a result ofmeasurement from the electric field strength sensor 21, and outputs theresult to the position detector circuit 75.

The position detector circuit 75 detects a present position of thecapsule endoscope 11 inside the test body on the basis of the fieldstrength of the radio wave 14 that is measured by the electric fieldstrength sensor 21, and the position detector circuit 75 outputsinformation on the detected position of the capsule endoscope 11,hereinafter referred to as imaging position data, to the data storage73. The data storage 73 records the imaging position data in associationwith the CE image data that is obtained through the capsule endoscope 11at the imaging position represented by the imaging position data. Sincethe method of detecting the position of the capsule endoscope 11 insidethe test body on the basis of the field strength of the radio wave 14from the capsule endoscope 11 is well known in the art, details of thismethod are omitted from the present description.

As shown in FIG. 5, the first processor 24 of the workstation 13includes a CPU 83 that supervises the overall operation of theworkstation 13. A data bus 84 connects the CPU 83 to a RAM 86, acommunication interface (I/F) 87 and a data storage 88. The data bus 84is connected to an LCD driver 89 for driving the LCD 26 and to a LANinterface (I/F) 90 that is connected to the LAN 29.

The CPU 83 reads necessary programs and data out of the data storage 88and expands them on the RAM 86 to process the read programssequentially. The CPU 83 also controls the respective components of theworkstation 13 to work in accordance with operational signals inputthrough the operating members 25. The communication interface 87intermediates data exchange between the workstation 13 and the receiver12 through a USB connector 91, and receives the CE image data and theimaging position data from the receiver 12. The received CE image dataand imaging position data are stored in the data storage 88.

As shown in FIG. 6, the data storage 88 has an image data storagesection 93, an imaging position data storage section 94, a programstorage section 95 and an image characteristic value storage section 96.The image data storage section and the imaging position data storagesection 94 store the CE image data and the imaging position datarespectively while sorting out the data for one patient from another.

The program storage section 95 stores a point image selection program 97beside various programs and data for controlling the operation of thefirst processor 24. When the point image selection program 97 isactivated, a point image selection screen 98 (see FIG. 8) is displayedon the LCD 26, allowing selecting images from among those images whichcorrespond to the CE image data fed through the receiver 12.

As shown for example in FIG. 7, if a doctor finds a CE image containingsuch a questionable portion that can be a lesion as a result of theimage interpretation on the LCD 26, the doctor selects the CE imagecontaining the questionable portion as a CE image of an aimed point.Thereafter, the doctor selects at least a second CE image that was takenon the way from an inserting position of the capsule endoscope 11 intothe patient 10 to the aimed point. In the example shown in FIG. 7, boththe capsule endoscope 11 and the balloon endoscope 31 are insertedthrough the mouth 10 a into the digestive tract of the patient 10, andintermediate points along a route R of insertion of the balloonendoscope 31, which corresponds to the course of movement of the capsuleendoscope 11 in this embodiment, are selected serially as first toZ^(th) pass points (Z is an integer larger than 1). Although the presentembodiment will be described on the assumption that the aimed point andthe pass points are located in the small intestines, these points can belocated in any positions inside the test body.

The probing tip 37 a of the balloon endoscope 31 enters through themouth 10 a and reaches the aimed point through the first to the Z^(th)pass points. In the following description, a CE image of the aimed pointwill be called an aimed point image, and CE images of the first toZ^(th) pass points will be called first to Z^(th) pass point imagesrespectively.

The aimed point image and the first to Z^(th) pass point images areutilized for detecting which point the probing tip 37 a has reached onthe way to the aimed point during the endoscopy with the balloonendoscope 31. As described above, the balloon 48 of the balloonendoscope 31 draws the small intestines in, to thrust the inserter 37 ofthe balloon endoscope 31 through into the small intestines. The BE imagedata obtained by the balloon endoscope 31 is compared with the CE imagedata obtained at the respective pass points and the aimed point, tocheck similarity between them. Based on the similarity between them, itis possible to detect which point the probing tip 37 a has reached. Notethat the similarity between the BE image data and the CE image data isjudged by image characteristic values that are extracted from therespective image data.

Referring back to FIG. 5, a distortion correcting processor (expandedimage producing device) 100, a spectral image producer (first spectralimage producer) 101 and an image characteristic value extractor (firstimage characteristic value taking device) 102 are built up in the CPU 83when the point image selection program 97 is activated.

The distortion correcting processor 100 processes image data of those CEimages which the doctor selects by operating the operating members 25,for the sake of correcting trapezoidal distortion, called keystonecorrection. As described above, the objective lens 59 of the capsuleendoscope 11 (see FIG. 3) forms an omniazimuth image of a viewed bodysite, so the CE image data is omniazimuth image data. Since the BE imageis a planer image, it is necessary to expand the CE image to be a planerimage so as to detect similarity or coincidence between the BE image andthe CE image.

For this purpose, the distortion correcting processor 100 subjects theimage data of those CE images which are selected by the doctor to akeystone correction process to produce the expanded image data. Therebythe image data of the respective points are converted to expanded imagedata. It is possible to produce the expanded image data by other knowndistortion correction method or image expansion method than the keystonecorrection.

Te 101 produces spectral image data of an arbitrary spectral frequencyband (wavelength band) from the expanded image data that is produced bythe distortion correcting processor 100. In order to detect exactlywhether the probing tip 37 a reaches the selected points during theendoscopy with the balloon endoscope 31, it is desirable that the pointimage data is definitely different from the CE image data of peripheralareas around the selected points. If the point image data of one pointis similar to the CE image data of the peripheral area around thatpoint, a mistake is likely to occur that the probing tip 37 a isconsidered to reach the destination point before the probing tip 37 areaches that point.

To prevent such an error, the spectral image producer 101 producesspectral image data (first spectral image) from the expanded image dataas produced by the distortion correcting processor 100. Thus, everypoint image data of the respective images selected by the doctor isconverted to the expanded image data and then to the spectral imagedata. In the field of medical diagnosis with the endoscopy, the spectralimages of the targets are often produced to facilitate finding outlesions, because the spectral images can enhance appropriate features ofthe targets, e.g. blood vessels or some organs such as stomach innerwalls and bowel surface tissues, without the need for coloring thetargets. Producing the spectral image data of the respective points toenhance some features of these points, e.g. blood vessels, is increasingthe difference between the point image data and the CE image data of theperiphery of the selected point. Since the spectral frequency band thatis effective to increase the difference between the point image data andthe peripheral CE image data varies depending upon the target organ, thespectral image producer 101 is designed to produce the spectral imagedata of a variable spectral frequency band, which is adjustable throughthe operating members 25.

When an operation for displaying a spectral image is done on theoperating members 25, the spectral image producer 101 reads outcoefficient matrix (not shown) from the data storage 88 or anothermemory location, to execute a matrix calculation for multiplying theexpanded image data with coefficients of the matrix, thereby to producethe spectral image data. Since the method of producing the spectralimage data using a coefficient matrix is well-known in the art and isdisclosed for example in JPA 2007-319442, the description of this methodis omitted here. It is possible to use another method for producing thespectral image data.

The image characteristic value extractor 102 extracts characteristicvalues from the spectral image data of the first to the Z^(th) passpoints and the spectral image data of the aimed point. Hereinafter, thespectral image data of the pass points and that of the aimed point willbe referred to as the pass point spectral image data and the aimed pointspectral image data. The characteristic values of the aimed pointspectral image data as well as the pass point spectral image data, whichcorrespond to the first mentioned first image characteristic values,represent numerical information on characteristic values of theindividual images, including total hue, color distribution, contourdistribution and shape of each image. These characteristic values areserved to calculate similarity of the BE image data to the image. In thepresent embodiment, image characteristic values representative ofpatterns of blood vessels in the individual point image are extracted.

The image characteristic values representative of the blood vesselpatterns may be “distribution of edge directions of blood vessels”,“distribution of curvatures of blood vessels”, “the number of vascularbranch-off points and their relative locations” and “variation patternin the edge directions of blood vessels”. The distribution of edgedirections of blood vessels means a distribution of the directions ofedges (lateral or radial ends of contours) of blood vessels.Specifically, all blood vessels contained in the spectral images of theaimed point and the respective pass points, which are based on the pointspectral image data, are subdivided into segments of constant lengths,and the direction of each segment of the blood vessel is detected as anangle (0° to 180°) to a reference direction that may be predeterminedappropriately. Thus, the distribution of edge directions representsdistribution of the directions of the respective blood vessel segments.

The distribution of curvatures represents a distribution of curvaturesof the respective segments of all blood vessels contained in thespectral images of the respective points. The number of vascularbranch-off points and their relative locations represent the number ofbranch-off points of the blood vessels in each spectral image of theselected points and the relation in location between these branch-offpoints. The relation in location between the branch-off points may forexample be expressed in an XY coordinate system, of which a referencepoint (0, 0) is defined in the image at an appropriate one of the branchpoints. The variation pattern in the direction of vascular edgesrepresents variation patterns of the blood vessels contained in thespectral images of the respective points. As an example of a variationpattern, a blood vessel spreads into two branches and the two branchesfurther spreads into two and three directions respectively.

The method of extracting image characteristic values of the blood vesselpattern may be a conventional one, so the description of the bloodvessel pattern extracting method is omitted here. Data of the pointimage characteristic values extracted by the image characteristic valueextractor 102 are accompanied with additional information on spectralfrequency band of the point spectral image data and on file name ofimage files of the respective point images as the originals of therespective point spectral images. The spectral frequency band data isobtained from the spectral image producer 101, whereas the file namedata is obtained from the image data storage section 93.

The image characteristic value extractor 102 outputs the characteristicvalues of the respective point images, which are extracted from thepoint spectral image data, to the data storage 88. Then the imagecharacteristic value storage section 96 of the data storage 88temporarily stores the respective point image characteristic valueswhile sorting them according to the patients (see FIG. 6).

FIG. 8 shows the point image selection screen 98, which serves forselecting the point images from the CE images that are taken through thereceiver 12. The point image selection screen 98 is provided with apatient selection box 105, an image display window 107, a spectralfrequency band input box 108 and a selection result display window 109.

The patient selection box 105 is used for selecting a patient 10 as asubject of an image interpretation, and the CE images obtained by thecapsule endoscope 11 from the selected patient are read out from thedata storage 88. When a pointer 110 is put on a mark of an invertedtriangle at the right end of the patient selection box 105 and the mouseis clicked, a list of patients' names and ID numbers is displayed in theform of a pull-down menu. By clicking on one of the patients' names, theone patient is selected as the subject.

The image display window 107 displays the CE images of the patient 10 asselected in the patient selection box 105. For example, the CE imagesare seriatim displayed on the image display window 107 in the same orderas the time sequence of imaging by the capsule endoscope 11. The doctorcan control playing, pausing and quitting the display on the imagedisplay window 107 by clicking corresponding marks on a control bar 107a. Thus, the doctor interprets or investigates the CE images one by onewhile they are being displayed sequentially.

When the doctor does not find any portion that can be a lesion in thedisplayed CE image, the doctor proceeds to the next CE image withoutmaking any particular operation. If the doctor find a suspected portionthat can be a lesion in the displayed CE image, the doctor pauses thedisplay to investigate that CE image in more detail. Hereinafter the CEimage containing the suspected portion will be referred to as the aimedpoint candidate image. With the CE image, its file name and imagingposition data are displayed on an upper zone of the image display window107. If the doctor judges it necessary to make a thorough examination ofthe site that is captured as the aimed point candidate image, the doctormakes an operation for producing a spectral image of the aimed pointcandidate image.

Specifically, the doctor inputs a spectral frequency band in thespectral frequency band input box 108, for producing the spectral image,while the aimed point candidate image is being paused on the imagedisplay window 107. Next, the doctor clicks the pointer 110 on aspectral image display button 111. Thereby, an expanded image data isproduced by expanding the aimed point candidate image data, and thenspectral image data having the spectral frequency band input by thedoctor is produced from this expanded image data. Based on the producedspectral image data, a spectral image is displayed on the image displaywindow 107.

When the doctor changes the spectral frequency band in the spectralfrequency band input box 108 and thereafter clicks the spectral imagedisplay button 111, a spectral image having the changed spectralfrequency band is displayed on the image display window 107. Thus, thedoctor may repeat the same operation till a most desirable spectralimage, e.g. one with an enhanced blood vessel pattern, is displayed onthe image display window 107. When the desirable spectral image isdisplayed, the doctor clicks the pointer 110 on a point selection button112. Then the spectral image data of the displayed spectral image isselected as the aimed point image data. Simultaneously, imagecharacteristic values are extracted from the aimed point image data, andthe extracted characteristic values of the aimed point are stored in theimage characteristic value storage section 96 of the data storage 88(see FIG. 6).

The selection result display window 109 displays file names andrespective spectral frequency bands of those spectral images which areselected as the point images by the doctor. For example, when the aimedpoint image data is selected, a file name and a spectral frequency bandof the selected image data are displayed in a raw of “aimed point” onthe selection result display window 109.

After selecting the aimed point image data, the doctor selects imagedata of the respective pass points on the point image selection screen98. First, those CE images are seriatim displayed on the image displaywindow 107, which are captured from the selected patient 10 on the wayfrom the mouth 10 a or the inlet for the inserter 37 of the balloonendoscope 31 to the aimed point, in the same order as the time sequenceof imaging. The doctor observes the successive CE images and pauses thedisplay by operating the control bar 107 a when such a CE image isdisplayed that can be the first pass point image, hereinafter referredto as the first pass point candidate image. At that time, a file nameand imaging position data of the first pass point candidate image aredisplayed on the upper zone of the image display window 107.

The doctor checks the first pass point candidate image and its file nameand imaging position data on the image display window 107, to decidewhether the candidate image is to be the first pass point image or not.If the doctor does not decide the candidate image to be the first passpoint image, the doctor operates the control bar 107 a to restartdisplaying the CE images in succession. When the doctor decides on thefirst pass point image, the doctor inputs a spectral frequency band forthe first pass point image in the spectral frequency band input box 108and clicking the spectral image display button 111. Thereby, the CEimage data of the first pass point image is expanded, and spectral imagedata having the designated spectral frequency band is produced from theexpanded image data. Based on the spectral image data, a spectral imageis displayed on the image display window 107.

The doctor repeats inputting a spectral frequency band and checking aspectral image having the input spectral frequency band on the imagedisplay window 107 till the displayed spectral image contain an enhancedblood vessel pattern that facilitates discriminating the first passpoint image from others. When a desirable spectral image is displayed onthe image display window 107, the doctor clicks on the point selectionbutton 112, upon which a file name and a spectral frequency band of theselected spectral image are displayed in a raw of “first pass point” onthe selection result display window 109. This means that the image dataof the first pass point spectral image as displayed on the image displaywindow 107 is selected as the first pass point image data.

Simultaneously with the selection of the first pass point image data,image characteristic values are extracted from the first pass pointimage data, and the extracted characteristic values of the first passpoint are stored in the image characteristic value storage section 96 ofthe data storage 88, completing the selection of the first pass pointimage data.

Image files of the second and following pass point spectral images areselected in the same way as for the first pass point, and imagecharacteristic values of the second and following pass points are storedin the image characteristic value storage section 96. When completingselecting all necessary points and their spectral images, the doctorclicks the pointer 110 on a selection complete button 113. Then theimage characteristic values of the respective point images aretransmitted from the image characteristic value storage section 96 tothe second processor 32 through the LAN interface 90 and the LAN 29.

Now the balloon endoscope 31 and the second processor 32, whichconstitute the electronic endoscopy system 4, will be described indetail. As shown in FIG. 9, an optical image of a subject or target siteis formed through the objective lens 41 on an imaging surface of theimaging device 42 of the balloon endoscope 31, and the imaging device 42outputs analog picture signals from its respective pixels to an analogfront end (AFE) circuit 115. The AFE circuit 115 treats the picturesignals with correlated double sampling, amplification andanalog-to-digital conversion, to convert them to digital BE image data.The AFE circuit 115 outputs the BE image data to the second processor 32through the universal cord 39. It is alternatively possible to mount theAEF circuit 115 in the second processor 32 and convert the picturesignals output from the imaging device 42 to the BE image data in thesecond processor 32.

A CPU 116 of the second processor 32 controls the overall operation ofthe electronic endoscopy system 4. A data bus 117 connects the CPU 116to a RAM 119, a digital signal processor (DSP) 120, an image memory 121and a data storage 122. The data bus 117 is also connected to an LCDdriver 123 for driving the LCD 35 and a LAN interface (I/F) 124 that isconnected to the LAN 29. The LAN interface 124 is supplied with theimage characteristic values of the respective point images, which arefed from the first processor 24 through the LAN 29. The imagecharacteristic values are stored in the data storage 122.

The CPU 116 reads necessary programs and data from the data storage 122and expands them on the RAM 119 to process the read programssequentially. The CPU 116 also controls the respective components of thesecond processor 32 to work in accordance with operational signals inputthrough the operating members 34. The digital signal processor 120processes the BE image data fed from the balloon endoscope 31. Theprocessed BE image data is temporarily stored in the image memory 121.The image memory 121 overwrites the previously stored BE image data withthe new BE image data as it is fed from the digital signal processor120.

The LCD driver 123 is connected to a not-shown VRAM, which stores the BEimage data as being read out from the image memory 121. Writing andreading of the BE image data in and out of the VRAM is being carried outin parallel to each other. The LCD driver 123 converts the BE imagedata, as being read out from the VRAM, to an analog composite signal fordisplaying the BE image on the LCD 35.

The data storage 122 has an image characteristic value storage section126 and a program storage section 127. The image characteristic valuesof the respective point images, which are fed from the first processor24, are stored in the image characteristic value storage section 126while being sorted out them for one patient from another. Based on theimage characteristic values of the respective point images, the CPU 116detects which point the probing tip 37 a has got to among the respectivepass points and the aimed point while the inserter 37 of the balloonendoscope 31 is being inserted into the patient 10 for the sake of adetailed examination of the aimed point. Concretely, the CPU 116calculates the degree of similarity between image characteristic valuesof the BE image data and the stored image characteristic values of therespective point images.

The program storage section 127 stores a point detection program 128beside various programs and data for controlling the operation of thesecond processor 32. When the point detection program 128 is activated,a point detection screen 129 (see FIGS. 10 and 11) is displayed on theLCD 35, showing which point the probing tip 37 a of the balloonendoscope 31 has reached. When the point detection program 128 isactivated, a spectral image producer (second spectral image producer)131, an image characteristic value extractor (second imagecharacteristic value taking device) 132 and a point detector (positioninformation obtaining device) 133 are built up in the CPU 116.

The spectral image producer 131 reads out the BE image data from theimage memory 121 and produces BE spectral image data (second spectralimage) from the read BE image data, so that the BE spectral image datahas the same spectral frequency band as the first spectral image data ofthe destination point has on the basis of the image characteristicvalues (spectral frequency band data) of the destination point that theprobing tip 37 a of the balloon endoscope 31 is approaching.

For example, when the destination point is the first pass point, thespectral image producer 131 retrieves the spectral frequency band dataof the first pass point image characteristic values from the imagecharacteristic value storage section 126 of the data storage 122. On thebasis of the read spectral frequency band data, the spectral imageproducer 131 produces the BE spectral image data having the samespectral frequency band as the first spectral image data of the firstpass point has. As for the second and following pass points as well asthe aimed point, the spectral image producer 131 produces BE spectralimage data having the same spectral frequency band as the first spectralimage data of the respective points has before the probing tip 37 areaches these points.

The BE spectral image data produced by the spectral image producer 131is temporarily stored in a not-shown spectral image data memory locationof the RAM 119. The BE spectral image data in the spectral image datamemory location of the RAM 119 is revised each time a new set of BEspectral image data is written therein.

The image characteristic value extractor 132 extracts imagecharacteristic values of the BE spectral image, hereinafter referred toas BE image characteristic values which correspond to thefirst-mentioned second image characteristic values, from the BE spectralimage data written in the spectral image data memory location of the RAM119. The BE image characteristic values represent image characteristicvalues of a blood vessel pattern in the observed site, like theabove-mentioned point image characteristic values. The BE imagecharacteristic values extracted by the image characteristic valueextractor 132 is temporarily stored in a not-shown image characteristicvalue memory location of the RAM 119. The image characteristic valueextractor 132 extracts the BE image characteristic values each time anew set of BE spectral image data is written in the RAM 119. The BEimage characteristic value data in the image characteristic value memorylocation of the RAM 119 is revised each time a new set of BE imagecharacteristic value data is written therein.

The point detector 133 calculates the degree of similarity between theBE image characteristic values as stored in the RAM 119 and the pointimage characteristic values of the destination point as stored in theimage characteristic value storage section 126, which is the CE spectralimage data obtained by the capsule endoscope 11 at the destinationpoint. Based on the degree of similarity, the point detector 133 judgeswhether the BE spectral image data obtained by the balloon endoscope 31is similar to the point image data of the destination point. That is,the point detector 133 judges whether the probing tip 37 a of theballoon endoscope 31 reaches the destination point.

As a formula for calculating the degree of similarity, the pointdetector 133 uses such a function that has the larger value as thedegree of similarity between two comparatives gets the higher. Forexample the point detector 133 uses the following formula:

D=C −Σ{ai×(vxi−vsi)²}

wherein vx is image characteristic values of the destination point, vsis BE image characteristic values, ai is weighting coefficient forrespective parameters, and i is parameter number.

The point detector 133 compares the calculated degree of similarity witha predetermined threshold value. When the calculated degree ofsimilarity is less than the predetermined threshold value, the pointdetector 133 judges that the BE image data is not similar to the pointimage data of the destination point, and that the probing tip 37 a ofthe balloon endoscope 31 does not reach the destination point. If thecalculated degree of similarity is equal to or more than thepredetermined threshold value, the point detector 133 judges that theprobing tip 37 a of the balloon endoscope 31 has reached the destinationpoint.

However, even when the balloon endoscope 31 and the capsule endoscope 11capture images from the same point, if the images have differentpostures from each other at that time, the posture of the BE imagediffers from that of the CE image. In that case, the BE image maycoincide with the CE image when the BE image rotates through anappropriate angle, e.g. 180 degrees. Without correcting the posture ofthe BE image, the calculated degree of similarity can be less than thepredetermined threshold value, even if the probing tip 37 a of theballoon endoscope 31 actually reaches the destination point. To avoidthis problem, it might be possible to lower the threshold value, whichincreases the probability of misjudging that the probing tip 37 a of theballoon endoscope 31 reaches the destination point, while the probingtip 37 a does not actually reach the destination point.

To solve this problem, according to the present embodiment, a pluralityof sets of rotational spectral image data are produced from the BEspectral image data while changing the posture of the image to differentdirections. From these different sets of rotational spectral image data,respective rotational image characteristic values are extracted. Thenthe point detector 133 calculates the degrees of similarity between therespective rotational image characteristic values and the point imagecharacteristic values of the destination point as well as the BE imagecharacteristic values and the point image characteristic values of thedestination point. If at least one of these calculated degrees ofsimilarity is equal to or more than the threshold value, the pointdetector 133 judges that the probing tip 37 a of the balloon endoscope31 reaches the destination point.

When it is judged that the probing tip 37 a reaches the destinationpoint, the next point is set to be the destination point. That is, whenthe probing tip 37 a is judged to reach the first pass point, the secondpass point is set to be the destination point. When the probing tip 37 ais judged to reach the last pass point, the aimed point is set to be thedestination point. The respective components 131 to 133 of the CPU 116produces the BE spectral image data, extracts the BE imagecharacteristic values, and calculates the degree of similarity on thebasis of the point image characteristic values of the newly set nextpoint. When the probing tip 37 a is judged to reach the aimed point, therespective components 131 to 133 of the CPU 116 terminate theprocessing.

As shown in FIGS. 10 and 11, the point detection screen 129 displaysinformation on which point the probing tip 37 of the balloon endoscope31 has reached among the first to Z^(th) pass points and the aimedpoint. The point detection screen 129 is provided with an image displaywindow 135, a destination point display window 136, a spectral frequencyband display window 137 and a message display window 138.

The image display window 135 displays a BE image presently obtained bythe balloon endoscope 31. It is possible to display a BE spectral imageon the basis of the BE spectral image data, simultaneously with the BEimage. The destination point display window 136 displays the name of thedestination point, e.g. the first pass point. The spectral frequencyband display window 137 displays information on the spectral frequencyband that is attached to the data of the point image characteristicvalues of the destination point.

The message display window 138 displays a message that informs ofwhether the probing tip 37 a of the balloon endoscope 31 has reached thedestination point. For example, while the destination point is the firstpass point and the probing tip 37 a has not reached the first passpoint, a message “not reached the first pass point” is displayed in themessage display window 138, as shown for example in FIG. 10.

When the probing tip 37 a has reached the first pass point, a message“reached the first pass point” is displayed in the message displaywindow 138, as shown for example in FIG. 11. At that time, the secondpass point is set to be a new destination point, so the destinationpoint display window 136 displays “the second pass point”, and thespectral frequency band display window 137 displays the spectralfrequency band data that corresponds to the point image characteristicvalues of the second pass point. When the probing tip 37 a has passedthe first pass point, the message in the message display window 138 isrevised to an appropriate one, like “not yet reached the second passpoint”.

In the same way as above, while the probing tip 37 a of the balloonendoscope 31 is moving toward the second and following pass points andthe aimed point, the name of the destination point, the spectralfrequency band data on the spectral image of the destination point, anda corresponding message are displayed in the respective windows 136 to138. So the doctor may check if the probing tip 37 a of the balloonendoscope 31 gets to the destination point. If the probing tip 37 a doesnot get to the destination point, the doctor can move the probing tip 37a forward or backward to reach the destination point. This way, theprobing tip 37 a of the balloon endoscope 31 finally gets to the aimedpoint.

Now the operation of the endoscopy system 2 as configured above will bedescribed. In the endoscopy system 2, the patient 10 first gets anendoscopy with the capsule endoscope 11 (the capsule endoscopy system3), and if the result of the capsule endoscopy shows any suspectedportion, the patient 10 gets an endoscopy with the balloon endoscope 31(the electronic endoscopy system 4) to investigate the suspected portionin detail.

First, the procedures in the capsule endoscopy system 3 will bedescribed with reference to FIG. 12. As a preparation for the endoscopy,the patient 10 puts the shield shirt 19 and the receiver 12 thereon.Then the patient 10 swallows the capsule endoscope 11 after its powerswitch being turned on. As the capsule endoscope 11 goes through thepatient's tract, it captures images from internal surfaces of the tract,and sequentially transmits the CE image data of the captured images inthe form of the radio wave 14. The radio wave 14 is received on theantennas 20. Simultaneously, the field strength of the received radiowave 14 is detected by the electric field strength sensor 21 that ismounted to each antenna 20. The detection results from the respectivesensors 21 are input to the position detector circuit 75 of the receiver12.

The radio wave 14 received on the antennas 20 is sent through thereceiver circuit 81 to the demodulator 71, to be demodulated to theoriginal CE image data. The CE image data is processed in the imageprocessing circuit 72 and output to the data storage 73. The positiondetector circuit 75 detects the present position of the capsuleendoscope 11 in the patient 10 on the basis of the detection result fromthe electric field strength sensor 21, and outputs the data of thepresent position of the capsule endoscope 11, i.e. the imaging positiondata, to the data storage 73. The data storage 73 stores the imagingposition data in association with the image data that is fed from theimage processing circuit 72.

When the imaging or endoscopy with the capsule endoscope 11 is complete,the receiver 12 is connected to the first processor 24 through the USBcable 27. Next, the doctor operates the operating members 25 to transferthe CE image data and the imaging position data from the data storage 73to the first processor 24. Then the CE image data is stored in the imagedata storage section 93 of the data storage 88, and the imaging positiondata is stored in the imaging position data storage section 94 of thedata storage 88. After the all data is transferred from the data storage73 to the data storage 88, the doctor operates the operating members 25to activate the point image selection program 97, thereby displaying thepoint image selection screen 98 on the LCD 26.

The doctor operates the pointer 110 to select the patient 10 in thepatient selection box 105 on the point image selection screen 98. TheCPU 83 reads out the CE image data of the selected patient from theimage data storage section 93, to display the CE images of this patientsuccessively on the image display window 107. If the displayed CE imagedoes not contain any suspected portion that looks like a lesion, thedoctor has the next CE image displayed to investigate it. If the doctordoes not find any suspected portion in any of the CE images that hasbeen transferred from the receiver 12, the doctor closes the point imageselection screen 98 to terminate the point image selection process.

On the contrary, if the doctor finds such a portion that is suspected tobe a lesion in the CE image displayed in the image display window 107,the doctor operates the control bar 107 a to pause the displayed CEimage, and investigates the displayed CE image as an aimed pointcandidate image. If the doctor judges it unnecessary to make a thoroughexamination of the suspected portion as contained in the aimed pointcandidate image, the doctor operates the control bar 107 a to restartthe successive display of the CE images. If the doctor judges itnecessary to make a thorough examination of the suspected portion ascontained in the aimed point candidate image with the balloon endoscope31, the doctor operates the operating members 25 to input an appropriatespectral frequency band in the spectral frequency band input box 108 andclicks the spectral image display button 111.

Upon the spectral image display button 111 being clicked on, thedistortion correcting processor 100 of the CPU 83 produces expandedimage data from image data of the aimed point candidate image, and fromthe expanded image data, the spectral image producer 101 producesspectral image data of the input spectral frequency band that is inputin the spectral frequency band input box 108. Then the CPU 83 displays aspectral image in the image display window 107 on the basis of thespectral image data produced by the spectral image producer 101.

The doctor repeats the above operation till a desirable spectral image,e.g. one having an enhanced blood vessel pattern, is displayed. When thedesirable spectral image is displayed, the doctor clicks the pointselection button 112. Thereby, the file name and the spectral frequencyband of the selected image are displayed in the raw of “aimed point” ofthe selection result display window 109. Thus, the spectral image dataof the aimed point is selected. Simultaneously, the image characteristicvalue extractor 102 extracts image characteristic values of the aimedpoint from the spectral image data of the aimed point, and outputs thesevalues to the data storage 88. Then the image characteristic values ofthe aimed point are temporality stored in the image characteristic valuestorage section 96.

When the selection of the spectral image data of the aimed point iscomplete, the doctor begins to select pass points and spectral images ofthe pass points. When the doctor makes an operation for startingselecting pass point image data by the operating members 25 and thelike, the CPU 83 refers to the imaging position data stored in theimaging position data storage section 94, and lets those CE images,which have been taken from the patient 10 as the capsule endoscope 11moved from the mouth 10 a to the aimed point, be displayed successivelyin the image display window 107. Checking the successively displayed CEimages, the doctor pauses the display of such a CE image that can be afirst pass point image by operating the control bar 107 a.

Then the doctor checks the CE image displayed standstill in the imagedisplay window 107 and its imaging position data, to decide whether thiscandidate image should be selected to be a first pass point image. Whenthe doctor decides not to select the candidate image as the first passpoint image, the doctor restarts the successive display of the CEimages.

When the doctor decides to select the candidate image as the first passpoint image, the doctor inputs a spectral frequency band and clicks thespectral image display button 111. Then the distortion correctingprocessor 100 produces expanded image data from image data of the firstpass point candidate image, and from the expanded image data, thespectral image producer 101 produces spectral image data of the inputspectral frequency band. Then a spectral image is displayed in the imagedisplay window 107 on the basis of the spectral image data produced bythe spectral image producer 101. The doctor repeats the above operationtill a desirable spectral image, e.g. one having an enhanced bloodvessel pattern, is displayed.

When the desirable spectral image is displayed, the doctor clicks thepoint selection button 112. Thereby, the file name and the spectralfrequency band of the selected image are displayed in the raw of “firstpass point” of the selection result display window 109. Thus, thespectral image data of the first pass point is selected. Simultaneously,the image characteristic value extractor 102 extracts imagecharacteristic values from the spectral image data of the first passpoint, and outputs these values to the data storage 88. Then the imagecharacteristic values of the first pass point are temporality stored inthe image characteristic value storage section 96.

In the same way as for the first pass point, the second and followingpass point spectral images are selected, and image characteristic valuesof the second and following pass points are stored in the imagecharacteristic value storage section 96. When completing selecting allnecessary points and their spectral images, the doctor clicks theselection complete button 113. Then the image characteristic values ofthe respective point images are transmitted from the imagecharacteristic value storage section 96 to the second processor 32through the LAN 29, and are stored in the image characteristic valuestorage section 126 of the data storage 122. At each endoscopy with thecapsule endoscope 11, the above described operations and processes arecarried out.

Next, the doctor makes a thorough examination of the aimed point, whichis found by the endoscopy with the capsule endoscope 11. The procedureof the thorough examination using the electronic endoscopy system 4 willbe described with reference to FIG. 13. First the doctor inputsinformation on the patient 10 as the subject of the electronicendoscopy, connects the universal cord 39 of the balloon endoscope 31 tothe second processor 32 and the illuminator 33, and activates the pointdetection program 128 by operating the operating section 34.

When the point detection program 128 is activated, the point detectionscreen 129 is displayed on the LCD 35. The CPU 116 initially sets up thefirst pass point as the destination point, and reads out the imagecharacteristic values of the first pass point image of the subjectpatient 10 from the image characteristic value storage section 126 ofthe data storage 122. Then the name of the next destination point, thecorresponding spectral frequency band data and the message are displayedin the respective windows 136 to 138 of the point detection screen 129.

When the point detection screen 129 is displayed on the LCD 35, thedoctor turns the illuminator 33 on and inserts the inserter 37 of theballoon endoscope 31 through the mouth 10 a into the tract of thepatient 10, so that the imaging device 42 built in the probing tip 37 acaptures images from internal surfaces of the tract. Analog picturesignal output from the imaging device 42 is converted through the AFEcircuit 115 to digital image data. The BE image data obtained this wayby the balloon endoscope 31 is fed to the second processor 32 throughthe universal cord 39.

The BE image data is processed in the digital signal processor 120 ofthe second processor 32 and then stored in the image memory 121. Basedon the BE image data stored in the image memory 121, the BE image isdisplayed in the image display window 135 of the point detection screen129. While watching the BE image in the image display window 135, thedoctor advances the probing tip 37 a toward the first pass point bydrawing the patients' small intestines with the balloon 48 of theballoon endoscope 31.

Simultaneously, the spectral image producer 131 of the CPU 116 producesBE spectral image data having the same spectral frequency band as thespectral image data of the first pass point has on the basis of thespectral frequency band data included in the first pass point imagecharacteristic values, which has previously been read out from the imagecharacteristic value storage section 126. Then the image characteristicvalue extractor 132 extracts the BE image characteristic values from theBE spectral image data as produced by the spectral image producer 131.

Next, the point detector 133 calculates the degree of similarity betweenthe BE image characteristic values, which are extracted by the imagecharacteristic value extractor 132, and the first pass point imagecharacteristic values. If the calculated degree of similarity is lessthan the predetermined threshold value, the point detector 133 judgesthat the probing tip 37 a of the BE 31 has not reached the first passpoint. Then a message “not reached the first pass point” is displayed inthe message display window 138, so the doctor further advancing theprobing tip 37 a of the balloon endoscope 31 to reach the first passpoint.

When the calculated degree of similarity gets to the predeterminedthreshold value, the CPU 116 judges that the probing tip 37 a hasreached the first pass point, and a message “reached the first passpoint” is displayed in the message display window 138. So the doctor canconfirm that the probing tip 37 a has reached the first pass point.Since the small intestines are drawn in by the balloon 48 to move theprobing tip 37 a to the first pass point, the relative position of thefirst pass point in the patient 10 can often vary between the endoscopywith the capsule endoscope 11 and the endoscopy with the balloonendoscope 31. However, according to the present embodiment, the judgmentas to whether the probing tip 37 a has reached the first pass point ornot is done on the basis of the result of calculation about thesimilarity between the BE image characteristic values and the imagecharacteristic values of the first pass point image. Therefore,regardless of the variation in relative position of the first passpoint, it is possible to judge with precision that the probing tip 37 ahas reached the first pass point.

When the CPU 116 judges that the probing tip 37 a has reached the firstpass point, it sets the second pass point to be the next destinationpoint, and reads out the image characteristic values of the second passpoint image from the image characteristic value storage section 126.Then “the second pass point” is displayed as the name of the nextdestination point in the window 136, and the corresponding spectralfrequency band data and the message are displayed in the respectivewindows 137 and 138. On the basis of the image characteristic values ofthe second pass point image, the respective components 131 to 133 of theCPU 116 produce the BE spectral image data from the BE image dataobtained by the balloon endoscope 31, extract the BE imagecharacteristic values and calculate the degree of similarity between theBE image characteristic values and the second pass point imagecharacteristic values.

As the probing tip 37 a of the balloon endoscope 31 moves to the secondand following pass points and further to the aimed point, the sameprocedures as above: producing the BE spectral image data, extractingthe BE image characteristic values and calculating the degree ofsimilarity, are carried out on the basis of the BE image data obtainedby the balloon endoscope 31 and the point image characteristic values ofthe next destination point. Depending upon whether the degree ofsimilarity is more or less than the threshold value, the judgment ismade as to whether the probing tip 37 a gets to the destination point ornot. The result of judgment is displayed on the point detection screen129. Thus, the doctor can sequentially advance the probing tip 37 a tothe respective pass point and finally to the aimed point. When theprobing tip 37 a has reached the aimed point, the doctor accuratelyexamines the aimed point with the balloon endoscope 31.

As described so far, in the endoscopy system 2 of the present invention,an endoscopy with the capsule endoscope 11 is first made to find anaimed point to be examined in detail with the balloon endoscope 31.Among the CE images obtained by the capsule endoscope 11, thoserepresentative of the aimed point and the pass points are selected asthe point images. The balloon endoscope 31 is advanced to the aimedpoint while checking the similarity between each of the selected pointimages and the BE images obtained by the balloon endoscope 31. Thus,even though the probing tip 37 a of the balloon endoscope 31 is movedthrough the small intestines by drawing them in with the balloon 48, itis possible to detect exactly which point in the patient's body theprobing tip 37 a has reached. That is, it is possible to detect therelative position of the probing tip 37 a in the tract or smallintestines of the patient 10.

Producing the CE spectral image data from the point images of therespective points and judging similarity between each point image andthe BE image on the basis of the CE spectral image data and the BEspectral image data ensure exact detection of whether the probing tip 37a of the balloon endoscope 31 has reached the destination point. Thefact that the difference between image data captured at one point andimage data captured in the periphery of that point is enhanced in thespectral image data prevents the mistake of judging the probing tip 37 ato have reached the destination point while the probing tip 37 a is inthe periphery of the destination point.

Although the present invention has been described with respect to theabove embodiment where the inserter 37 of the balloon endoscope 31 isinserted through the mouth 10 a of the patient 10, the present inventionis applicable to a case where the inserter 37 of the balloon endoscope31 is inserted through an anus 10 b of the patient 10, as shown in FIG.14. In that case, an insertion route Ra of the balloon endoscope 31extends from the anus 10 b to the aimed point. Therefore, the first tothe Z^(th) pass points are selected in the order of retracing the courseof the capsule endoscope 11, in opposition to the first embodiment.

In the above embodiment, “distribution of edge directions of bloodvessels”, “distribution of curvatures of blood vessels”, “the number ofvascular branch-off points and their relative locations” and “variationpattern in the edge directions of blood vessels” are referred to as theimage characteristic values representative of the blood vessel patterns.But the image characteristic values are not to be limited to thesefactors. For example, since the branch structure of the blood vessels isa kind of fractal structure, it is possible to calculate fractaldimension values of blood vessel edges in each point image, and use thefractal dimension values as the image characteristic values. The fractaldimension value quantifies the level of complicity of a figure orstructure, the vascular edges in the present case. The greater thefractal dimension value, it represents the higher complicity level ofthe figure. Since many methods for calculating the fractal dimensionvalue are known, for example in JPA 2007-151608, the description thereofwill be skipped.

As another method of calculating a degree of similarity between a pointimage and a BE image, it is possible to binarize and then thin anexpanded image of the point image and the BE image to produce thinnedimages of the point image and the BE image, so as to calculate thedegree of similarity between the thinned images, which represent corelines of the blood vessels in the present case. Note that thebinarization is a process, whereby a density value for “white”, e.g.“1”, is assigned to those pixels having higher density values than apredetermined threshold value, while a density value for “black”, e.g.“0”, is assigned to those pixels having such density values that areequal to or less than the threshold value. The thinning is a process,whereby connected components of the image data are converted to linearstructures. Both the binarization and the thinning are well known, seefor example JPA 2007-117108 and JPA 2005-157902, the description ofthese process will be skipped.

In the above embodiment, the imaging position data, which represents theposition of the capsule endoscope 11 inside the test body, is detectedbased on the result of measurement of the electric field strength of theradio wave 14 from the capsule endoscope 11. But the imaging positiondata may be detected other ways. For example, the capsule endoscope 11may be provided with acceleration sensors for detecting accelerationdegrees of the capsule endoscope 11 in three axial directions. Duralintegration of these acceleration degrees will provide a travel distanceof the capsule endoscope 11, and the imaging position data may bedetected based on the travel distance.

Although the above-described embodiment produces the spectral image of asingle spectral frequency band from the expanded image data of eachpoint image, the present invention is not limited to this, but it ispossible to produce two or more spectral images of different spectralfrequency bands from the expanded image data of each point image. Inthat case, plural sets of spectral image data should be produced fromeach BE image, so as to have the corresponding spectral frequency bandsto the point spectral images respectively. Then the similarity betweenthe point spectral image data and the BE spectral image data of the samespectral frequency band is judged individually. Comparing the pointspectral images of variable frequency bands with the BE spectral imagesof the corresponding frequency bands contributes to improving accuracyof detection about whether the probing tip 37 a of the balloon endoscope31 has reached the destination point or not.

On producing the plural spectral images of variable frequency bands fromone CE image, it is possible to determine a set of different spectralfrequency bands or wavelength bands individually for each patient orcommonly to every patient. The set of different spectral frequency bandsmay be determined manually by the doctor or automatically according toinformation on the patient, such as the past illness and the suspecteddisease. It is possible to produce the spectral image data of differentspectral frequency bands automatically in the processor or the likeinstead of the manual production by the doctor.

Although the doctor selects the spectral frequency band appropriatelyfor each point to produce the point spectral image data from theexpanded image data of the CE point image obtained by the capsuleendoscope 11, the present invention is not to be limited to this. Forexample, it is possible to decide the spectral frequency band for thespectral image data according to the site or organ, e.g. stomach, smallbowel or large bowel, where the point image data was obtained. This isbecause every organ or site mostly has such a spectral frequency bandthat enhances vascular pattern in that organ or site the best.

Next, an endoscopy system according to another embodiment of the presentinvention will be described. Since the first embodiment does not detectwhich portion on the internal surface of the tract, i.e. the smallintestine in this case, the capsule endoscope 11 captured an image from,which is selected as a point image representative of a destinationpoint, i.e. a pass point or aimed point, it is sometimes difficult tofind the pass point or aimed point on the basis of the BE imagescaptured by the balloon endoscope 31. For example, when the point imageis an image of a front side portion of the internal wall of the smallintestine, the similarity between the point image and a BE image willnot be high if the balloon endoscope 31 captures the BE image from theopposite side, i.e. back side, even while the probing tip 37 a gets tothe destination point represented by the point image, i.e. the imagingposition of the capsule endoscope 11 for that point image.

To solve this problem, according to the second embodiment, not only therelative position of the capsule endoscope 11 inside the tract but alsothe attitude of the capsule endoscope 11, e.g. which side of the innerwall the objective lens 59 faces to, are detected and stored as imagingposition data. While the doctor advances the probing tip 37 a of theballoon endoscope 31 to a destination point, information on the attitudeof the capsule endoscope 11 at the imaging position for the destinationpoint is displayed on the LCD 35.

For this purpose, as shown in FIG. 3, the attitude sensor 140 isprovided in the capsule endoscope 11, to detect the attitude ororientation of the capsule endoscope 11 inside the tract of the testbody. The attitude sensor 140 may be any sensor insofar as it can detectthe attitude of the capsule endoscope 11. For example, the attitudesensor 140 may be a triaxial accelerometer or gravity sensor, or anattitude gyro. Since many methods of detecting the attitude of thecapsule endoscope 11 with an attitude sensor are known in the art, forexample from JP 3631265, JPA 2006-239053 and JPA 2006-068109, thedescription thereof will be omitted.

The detection result of the attitude sensor 140, i.e. the attitude data,is modulated with the CE image data into a radio wave 14 by themodulator circuit 54, and the radio wave 14 is sent from the sendercircuit 63 to the antenna 18. Thus, the attitude data is wirelesslytransmitted from the capsule endoscope 11 to the receiver 12. Thereceiver 12 stores the attitude data in the data storage 73 whileassociating it with the image data.

Like the first embodiment, the attitude data as well as the image datais fed to the first processor 24, and the doctor selects images of passpoints and an aimed point. Then, image characteristic values of theaimed point and the respective pass points are extracted and stored inthe image characteristic value storage section 96 of the data storage88. At that time, the attitude data representative of the attitude ofthe capsule endoscope 11 at each point is added to the point imagecharacteristic values of each point.

The point image characteristic values of the respective points,including the attitude data, are sent through the LAN 29 to the secondprocessor 32, and are stored in the image characteristic value storagesection 126 of the data storage 122. When the point detection program128 is activated at the start of endoscopy with the balloon endoscope31, a point detection screen 141 is displayed on the LCD 35, as shown inFIG. 16.

The point detection screen 141 may basically be the same as the pointdetection screen 129 of the first embodiment (see FIG. 10), except butthe point detection screen 141 has a different destination point displaywindow 142 from that of the first embodiment. Under the control of theCPU 116, the destination point display window 142 displays not only thename of the destination point but also information on the attitude ofthe capsule endoscope 11 at this destination point, on the basis of theattitude data added to the point image characteristic values of thispoint. For example, “front side” or “back side” or the like is displayedas the information on the attitude.

As the information on the attitude is displayed on the point detectionscreen 141, the doctor can see which side the destination point existson the inner wall of the small intestine. So the doctor may focus theprobing tip 37 a on the side where the destination point exits, whilethe doctor is advancing the probing tip 37 a toward the destinationpoint. It becomes easier for the doctor to get the probing tip 37 a tothe destination point.

Now an endoscopy system according to a third embodiment of the presentinvention will be described. While the first and second embodimentsextract image characteristic values representative of vascular patternsfrom the CE point images and the BE images, the third embodiment uses acapsule endoscope 145 and a balloon endoscope 146, which are differentfrom the endoscopes 11 and 31 of the above embodiments, and estimates ordetects information on asperities of the inner wall surface of a tract,e.g. a small intestine, from point image data and BE image data obtainedby these endoscopes 145 and 146. The information on the surfaceasperities is used as image characteristic values.

As shown in FIG. 17, the capsule endoscope 145 is fundamentallyconfigured the same way as the capsule endoscope 11, but a couple ofilluminator light sources 147R and 147L are disposed symmetrically to animaging device 60 in the capsule endoscope 145. The first and secondilluminator light sources 147R and 147L are made for example of LEDs.These light sources 147R and 147R are turned on alternately toilluminate the same subject, i.e. the same body portion, and a couple ofimages are captured from the same subject under the light from therespective light sources 147R and 147R. The speed of switching betweenthe light sources 147R and 147R is set so high as compared to thetraveling speed of the capsule endoscope 145 through the smallintestines that it is possible to capture images twice from the samesubject.

As the light sources 147R and 147R alternately illuminate the bowelinner wall, asperities of the bowel inner wall cast shadows. Since thefirst and second light sources 147R and 147R are apart from each other,the shadows casted on the inner wall of the small intestine by the firstlight source 147R vary in position and size from ones casted by thesecond light source 147L. Accordingly, pixels of one CE image CERcaptured under the light from the light source 147R have differentluminance values from the same pixels of the other CE image CEL have,which is captured from the same subject under the light from the secondlight source 147L.

Assuming that the light sources 147R and 147R are substantially at thesame distance from the inner wall of the small intestine, the respectiveluminance values of the pixels vary between the CE image CER and the CEimage CEL depending upon the asperities of the inner wall of the smallintestine, and more concretely the inclination of the inner wall of thesmall intestine. This fact will be explained with reference to anexample shown in FIGS. 18 and 19, where the imaging device 60 faces apeak P2 of a convex bowel inner wall portion S, so a point P1 opposingto the first light source 147R is on one side of the peak P2, and apoint P3 opposing to the second light source 147L is on the other sideof the peak P2. Note that the imaging device 60 and the light sources147R and 147R are assumed to be disposed on the same plane, to avoidcomplication of the drawing.

Providing that vR represents a luminance value of a pixel of the CEimage CER and vL represents a luminance value of a corresponding pixelof the CE image CEL, and that D represents a difference in luminancebetween the counterpart pixels of the CE images CER and CEL, thedifference D (=vR−vL) varies depending upon the relative positions ofthe respective light sources 147R and 147L to a point on the subjectthat corresponds to the counterpart pixels in the images CER and CEL.Specifically, since the first light source 147R has an incident anglecloser to a perpendicular to the subject surface at the point P1 thanthe second light source 147L, the light reflected (diffusively) from thepoint P1 is stronger under the light from the first light source 147R.Accordingly, as shown in FIG. 19, the luminance value of the pixelcorresponding to the point P1 is larger in the first CE image CER thanin the second CE image CEL, so the difference D gets a positive valuewith respect to the point P1. Since the two light sources 147R and 147Lhave approximately the same incident angle at the point P2, thedifference D is substantially zero. At the point P3, the second lightsource 147L has an incident angle closer to a perpendicular to thesubject surface at the point P1 than the first light source 147R, so thedifference D gets a negative value with respect to the point P3. Thedifference D gets the larger absolute value, the larger the inclinationangle of the subject surface becomes at the point P1 or P3. On thecontrary, where the subject surface of the bowel inner wall is concave,the difference D is substantially zero with respect to the point P2 thatis on the bottom the concave, and the difference D gets a negative valuewith respect to the point P1, whereas the difference D gets a positivevalue with respect to the point P3.

This way, it is possible to calculate the information on the asperitiesof the bowel inner wall from comparison between luminance values of thecounterpart pixels of CE images CER and CEL of each couple, regardingthe relative positions of the first and second light sources 147R and147L to the imaging device 60.

In the third embodiment, the balloon endoscope 146 has an observationwindow 149, first and second illumination windows 150R and 150L, anequipment outlet 151 and a gas/water nozzle 152 in its probing tip 37 a,as shown in FIG. 20. Behind the observation window 149 are disposed anobjective lens 41 and an imaging device 42, like as shown in FIG. 9. Theillumination windows 150R and 150L are placed symmetrically to theobservation window 149, to project illumination light from anilluminator 33 toward the bowel inner wall. The equipment outlet 151 andthe gas/water nozzle 152 are well known, so the description of thesemembers will be omitted.

The illuminator 33 is controlled by a control signal from a CPU 116 of asecond processor 32, to project illumination light alternately from thefirst illumination window 15OR and then from the second illuminationwindow 150L, and a couple of images BER and BEL are captured from thesame subject under the light from the respective illumination windows15OR and 150L. Therefore, in the same way as for the capsule endoscope145, it is possible to calculate the information on the asperities ofthe bowel inner wall from comparison between luminance values ofcounterpart pixels of the BE images BER and BEL of each couple,regarding the relative positions of the first and second illuminationwindows 150R and 150L to the observation window 149.

In the third embodiment, a CPU 83 of a first processor 24 and the CPU116 of the second processor 32 individually have a not-shown ridgeinformation estimator in place of the image characteristic valueextractor of the first embodiment. The ridge information estimator inthe CPU 83 obtains point image characteristic values by calculating theridge information from those CE images CER and CEL which the doctorselects as the point images. The ridge information estimator in the CPU116 obtains BE image characteristic values by calculating the ridgeinformation from the BE images BER and BEL captured by the balloonendoscope 146.

By checking the similarity between the point image characteristic valuesand the BE image characteristic values, it is possible to detect whichpoint the probing tip 37 a of the balloon endoscope 146 has reachedduring the endoscopy with the balloon endoscope 146, in the same way asdescribed with respect to the first embodiment. It is possible to obtainimage characteristic values representative of vascular patterns inaddition to the ridge information.

Although an optical axis O of the objective lens 59 is oriented parallelto a longitudinal axis of the capsule endoscope 145 in the thirdembodiment, so that light reflected from the subject enters through aface end of the capsule endoscope 145, the present invention is notlimited to this type of capsule endoscope. For example, as shown in FIG.21, such a capsule endoscope 154 is usable that an optical axis O of anobjective lens 59 is oriented perpendicular to a longitudinal axis ofthe capsule endoscope 154, and that light reflected from the subjectenters through an observation window formed on one side of the capsuleendoscope 154. Disposing first and second light sources 147R and 147L onopposite sides of an imaging device 60 symmetrically to the optical axisO in the capsule endoscope 154 enables estimating information onasperities of the bowel inner wall, in the same way as for the capsuleendoscope 145.

Although the first and second light sources 147R and 147L are disposedin the vicinity of the imaging device 60 in the capsule endoscopes 145and 154, it is possible to dispose a pair of light sources 147R and 147Lon opposite ends of a capsule endoscope 155, like as shown in FIG. 22.

Although the observation window 149 and the illumination windows 150Rand 150L are provided on a face end of the probing tip 37 a in theballoon endoscope 146 of the third embodiment, the present invention isnot limited to this. For example, it is possible to provide anobservation window 149 on one side of a probing tip 37 a of a balloonendoscope 156, and form first and second illumination windows 150R and150L symmetrically to the observation window 149, as shown in FIG. 23,wherein an equipment outlet and a gas/water nozzle are omitted from thedrawing for clarity sake. The same object as the balloon endoscope 146will be achieved by this configuration.

Although the above described capsule endoscopes 145, 154 and 155 have apair of illumination light sources that are disposed symmetrically tothe imaging device 60, the number and arrangement of the illuminationlight sources are not limited to the illustrated embodiments, but may bemodified appropriately. Likewise, the number and arrangement of theillumination windows of the balloon endoscope may be modifiedappropriately from those of the balloon endoscopes 146 and 156.Moreover, as for the capsule endoscopes 154 and 155, it is possible toturn the imaging device 60 and the light sources 147R and 147L together.The same applies to the observation window 149 and the illuminationwindows 150R and 150L of the balloon endoscope 156.

In the third embodiment, surface asperities of the subject (bowel innerwall) is estimated by comparing luminance values of the counterpartpixels of the CE images CER and CEL as captured from the same portion bythe capsule endoscope 145, and the information on surface asperities isused as the point image characteristic values. As another method ofdetecting image characteristic values of the point images, it ispossible to make a calculus of difference, such as disclosed in JPA2005-151099, for producing differential image data representative of adifference between the CE images CER and CEL of a particular point, andbinarize or thin the differential image data to extract information onshadows from the binarized or thinned differential image data. Theinformation on shadows may be used as the point image characteristicvalues of the particular point. In that case, information on shadows isextracted from the BE images BER and BEL of each couple, in the same wayas for the CE images, to use it as BE image characteristic values.

Furthermore, it is possible to change wavelength of illumination lightof the first light source 147R from that of the second light source147L, to detect information on surface asperities of an observed site onthe basis of a difference in color between the CE images CER and CEL. Inthis embodiment, illumination light from the first illumination window150R also has the different wavelength from that projected from thesecond illumination window 150L, to detect information on surfaceasperities of an observed site on the basis of a difference in colorbetween the BE images BER and BEL.

It is also possible to display the information on surface asperities orshadows of the CE image of the destination point in an enhanced manneron the BE image in an image display window 135 of a point detectionscreen 129 (see FIG. 10). Thereby, the doctor can visually see whetherthe probing tip 37 a of the balloon endoscope 146 has reached thedestination point or not.

Although the capsule endoscopy system 3 and the electronic endoscopysystem 4 have the individual processors 24 and 32 in the aboveembodiment, the present invention is not limited to this configuration,but the systems 3 and 4 have a common processor. In that case, the CEimage data may be transferred from the receiver 12 to the secondprocessor 32, so that the second processor 32 manages to accept thedoctor's selection of the respective point image data and extract thepoint image characteristic values. On the other hand, the firstprocessor 24 has only to serve for transferring the CE image data fromthe receiver 12 to the second processor 32.

In the above embodiment, the similarity of the BE images obtained by theballoon endoscope 31 to the point image as selected by the doctor isjudged by calculating a degree of similarity between imagecharacteristic values of the BE image and ones of the point image.However, the present invention is not limited to this method, but it ispossible to calculate directly a degree of similarity between thespectral image data of each point image and the spectral image of the BEimage. In that case, the spectral image data files of the respectivepoint images should be memorized in the second processor 32. The methodof calculating the similarity between image data files can be aconventional one, so the description will be omitted.

Although the above embodiment transmits the data of the point imagecharacteristic values from the first processor 24 to the secondprocessor 32 through the LAN 29, the present invention is not limited tothis, but the data may be transferred using various removable media.

Although the pass point images are selected by the doctor in the aboveembodiment, the present invention is not limited to this. For example,the CPU 83 may automatically select image data of several pass pointsfrom among those CE image data which are captured on the way from aninlet of the inserter 37 of the balloon endoscope 31, e.g. the patient'smouth, to an aimed point on the basis of the imaging position datastored in the imaging position data storage section 94, after the aimedpoint and its spectral image data are selected. The automatic selectionof the pass point images may be done in an appropriate manner, forexample, one out of a predetermined number of CE image frames (atconstant intervals), or once in a predetermined length of time forcapturing the CE images. From the automatically selected pass pointimages, the doctor may produce spectral images each individually, or awavelength parameter or spectral frequency band may automatically bedetermined for the spectral image of each pass point in the way asdescribed above.

Although the above embodiment is designed to select an aimed point imageand at least a pass point image from among the CE images captured by thecapsule endoscope 11, the present invention is not limited to this. Forexample, in a case where the aimed point is located near the inlet ofthe inserter 37 of the balloon endoscope 31, the doctor may have toselect the aimed point image alone.

In the first embodiment, the point image selection screen 98 (see FIG.8) is provided with the point selection button 112 for selecting theaimed point image data as well as the pass point image data, but it ispossible to provide an aimed point selection button and a pass pointselection button separately. Moreover, the point image selection screen98 may be provided with a window displaying a schematic illustration ofa human body like as shown in FIG. 7 that shows the selected points onthe basis of the imaging position data of the selected point image data,which are stored as the imaging position data of the CE images in theimaging position data storage section 94.

Although the above embodiments refer to the balloon endoscope for smallintestines as a flexible endoscope inserted in the test body, thepresent invention is not limited to this, but applicable to any casesusing other kinds of flexible endoscopes.

Thus, the present invention is not to be limited to the above embodimentbut, on the contrary, various modifications will be possible withoutdeparting from the scope of claims appended hereto.

1. An endoscopy system comprising: a capsule endoscope swallowed by atest body to capture first kind of images from internal portions of thetest body; an image processor for processing image data of the firstkind of images, to display the first kind of images on a monitor for adoctor to interpret them; a flexible endoscope having a flexibleinserter with an imaging device, said flexible inserter being insertedin the test body to capture second kind of images by said imaging devicewhen the doctor finds it necessary to make a thorough examination of anaimed point inside the test body as a result of interpretation of thefirst kind of images; an aimed point selection device for selecting anaimed point image that contains said aimed point from among the firstkind of images in response to an operation by the doctor; a similaritydetection device for detecting similarity between said aimed point imageand the second kind of images as captured by said imaging device whilesaid flexible inserter is being moved toward said aimed point; aposition information obtaining device for obtaining information on arelative position of said imaging device of said flexible endoscopeinside the test body on the basis of the similarity detected by saidsimilarity detection device; and a display device for displaying theinformation on the relative position of said imaging device.
 2. Anendoscopy system as recited in claim 1, further comprising a pass pointselection device for selecting at least a pass point image from amongthe first kind of images, said pass point image being representative ofa pass point on a route from an inlet of said flexible inserter to saidaimed point, wherein said similarity detection device further detectssimilarity between said pass point image and the second kind of imageswhile said flexible inserter is being inserted into the test body, andsaid position information obtaining device obtains the information onthe relative position of said imaging device to said pass point or tosaid aimed point on the basis of the similarity between said pass pointimage and the second kind of images or the similarity between said aimedpoint image and the second kind of images, respectively.
 3. An endoscopysystem as recited in claim 2, wherein said display device displaysinformation as to whether said imaging device has reached said passpoint or said aimed point as the information on the relative position ofsaid imaging device.
 4. An endoscopy system as recited in claim 2,further comprising: a first image characteristic value taking device fortaking image characteristic values respectively from said aimed pointimage and said pass point image; and a second image characteristic valuetaking device for taking image characteristic values from the secondkind of images, wherein said similarity detection device detects thesimilarity between said pass point image and the second kind of imagesby calculation using the image characteristic values of said pass pointimage and the second kind of image, and the similarity between saidaimed point image and the second kind of images by calculation using theimage characteristic values of said aimed point image and the secondkind of image.
 5. An endoscopy system as recited in claim 4, furthercomprising: a first spectral image producing device for producingspectral images of appropriately selected spectral frequency bands fromsaid aimed point image and said pass point image respectively; and asecond spectral image producing device for producing a spectral imagefrom each of the second kind of images so that said spectral image hasthe same spectral frequency band as said spectral image of said passpoint has while said imaging device of said inserter is moving towardsaid pass point, and that said spectral image has the same spectralfrequency band as the spectral image of said aimed point has while saidimaging device of said inserter is moving from said pass point towardsaid aimed point, wherein said first image characteristic value takingdevice takes the image characteristic values from said spectral imagesof said aimed point and said pass point, whereas said second imagecharacteristic value taking device takes the image characteristic valuesrespectively from said spectral images of the second kind of images. 6.An endoscopy system as recited in claim 4, wherein the imagecharacteristic values taken by said first and second imagecharacteristic value taking devices represent blood vessel patterns inthe internal portions of the test body.
 7. An endoscopy system asrecited in claim 4, wherein the image characteristic values taken bysaid first and second image characteristic value taking devicesrepresent surface asperities of the internal portions of the test body.8. An endoscopy system as recited in claim 7, wherein said capsuleendoscope comprises a number of light sources, which are disposed atdifferent positions and sequentially emit light to illuminate the sameportion inside the test body, and said capsule endoscope captures acorresponding number of images to the number of said light sources fromthe same portion synchronously with the sequential emissions of saidlight sources toward the same portion, and wherein said first imagecharacteristic value taking device estimates the surface asperities ofsaid pass point and said aimed point on the basis of images capturedfrom said pass point and images captured from said aimed pointrespectively.
 9. An endoscopy system as recited in claim 7, wherein saidinserter of said flexible endoscope is provided with a plurality ofillumination windows on different sides of said imaging device, toproject illumination light sequentially from one illumination windowafter another toward the same portion inside the test body, and saidimaging device of said flexible endoscope captures a correspondingnumber of images to said illumination windows from the same portionsynchronously with the sequential projection of illumination light fromsaid illumination windows toward the same portion, and wherein saidsecond image characteristic value taking device estimates the surfaceasperities of the same portion on the basis of the images captured bysaid imaging device of said flexible endoscope from the same portion.10. An endoscopy system as recited in claim 1, wherein the first kind ofimages as captured by said capsule endoscope are omniazimuth images, andthe second kind of images as captured by said flexible endoscope areplaner images, and wherein said similarity detection device expands thefirst kind of image of said aimed point to be a planer image andcompares the second kind of images with the expanded planer image ofsaid aimed point.
 11. An endoscopy system as recited in claim 2, whereinthe first kind of images as captured by said capsule endoscope areomniazimuth images, and the second kind of images as captured by saidflexible endoscope are planer images, and wherein said similaritydetection device expands the first kind of image of said pass point tobe a planer image and compares the second kind of images with theexpanded planer image of said pass point.
 12. An endoscopy system asrecited in claim 1, wherein said capsule endoscope is provided with anattitude detector for detecting attitude of said capsule endoscope, andwherein said endoscopy system further comprises a storage device forstoring information on the attitude of said capsule endoscope at saidaimed point in association with said aimed point image, and said displaydevice displays the information on the attitude of said capsuleendoscope at said aimed point besides the information on the relativeposition of said imaging device of said flexible endoscope to said aimedpoint.
 13. An endoscopy system as recited in claim 2, wherein saidcapsule endoscope is provided with an attitude detector for detectingattitude of said capsule endoscope, and wherein said endoscopy systemfurther comprises a storage device for storing data of the attitude ofsaid capsule endoscope at said pass point in association with said passpoint image, and said display device displays the information on theattitude of said capsule endoscope at said pass point besides theinformation on the relative position of said imaging device of saidflexible endoscope to said pass point.
 14. An endoscopy method using acapsule endoscope swallowed by a test body to capture first kind ofimages from internal portions of the test body, and a flexible endoscopehaving a flexible inserter with an imaging device, said flexibleinserter being inserted into the test body to capture second kind ofimages by said imaging device when the doctor finds it necessary to makea thorough examination of an aimed point inside the test body as aresult of interpretation of the first kind of images, said endoscopymethod comprising steps of: selecting an aimed point image that containssaid aimed point from among the first kind of images; detectingsimilarity between said aimed point image and the second kind of imagesas captured by said imaging device while said flexible inserter is beingmoved toward said aimed point; obtaining information on a relativeposition of said imaging device of said flexible endoscope inside thetest body on the basis of the similarity between said aimed point imageand the second kind of images; and displaying the obtained informationon the relative position of said imaging device.